---
id: running-playwright-performance-tests-with-artillery-and-tracetest
title: Performance Testing with Distributed Tracing using Artillery, Playwright and Tracetest
description: Quickstart on how to use the Tracetest Integration with Playwright and Artillery to enhance Performance Tests with Trace-Based Testing.
hide_table_of_contents: false
keywords:
  - tracetest
  - trace-based testing
  - observability
  - distributed tracing
  - testing
  - artillery
  - load testing
  - performance testing
  - e2e testing
  - playwright
  - end to end testing
  - end-to-end testing
  - integration testing
  - opentelemetry
image: https://res.cloudinary.com/djwdcmwdz/image/upload/v1698686403/docs/Blog_Thumbnail_14_rsvkmo.jpg
---

:::info Tracetest x Playwright Frontend Instrumentation Requirements
Find out the requirements for your instrumented app to start using [Tracetest x Playwright](https://tracetest.io/blog/tracetest-tip-instrumentation-for-end-to-end-tests).
:::

:::info Version Compatibility
This integration is compatible with [Artillery v2.0.10](https://github.com/artilleryio/artillery/releases/tag/artillery-2.0.10) and above.
:::

:::note
[Check out the source code on GitHub here.](https://github.com/kubeshop/tracetest/tree/main/examples/quick-start-artillery-playwright)
:::

[Tracetest](https://tracetest.io/) is a testing tool based on [OpenTelemetry](https://opentelemetry.io/) that permits you to test your distributed application. It allows you to use the trace data generated by your OpenTelemetry tools to check and assert if your application has the desired behavior defined by your test definitions.

[Playwright](https://playwright.dev/) is an open-source automation framework developed by Microsoft that enables cross-browser automation for web applications. It provides a set of APIs and libraries for automating interactions with web browsers such as Chrome, Firefox, and Microsoft Edge.

[Artillery](https://artillery.io/) is a modern, powerful load-testing toolkit. Artillery is designed to help developers and testers simulate traffic to their applications, APIs, and microservices. It allows users to define scenarios to test how their systems behave under different loads.

## Why is this important?

The Tracetest integration for Playwright enables your current Playwright tests to easily capture a full distributed trace from your OpenTelemetry instrumented front-end and back-end system.

You can embed a Tracetest test in this Playwright test, and allow trace-based testing assertions to be applied across this entire flow, enabling true end-to-end tests across your entire system. Taking it a step further you can use the Playwright engine in Artillery to run performance tests on top if it.

Because you're using traces as test specs you can:

- Get faster MTTR for failing performance tests
- Assert against the Artillery test execution, Playwright test execution, and the system under test
- Validate functionality of other parts of your system that may be broken, even when performance tests are passing


## The `@tracetest/playwright` npm Package

The [`@tracetest/playwright` npm package](https://www.npmjs.com/package/@tracetest/playwright) is a Playwright plugin that allows you to run trace-based testing using Tracetest and Playwright. It is a wrapper around the Tracetest API that allows you to configure, orchestrate and run tests from Playwright.

## The Artillery Playwright Engine

The [`artillery-engine-playwright` npm package](https://www.npmjs.com/package/artillery-engine-playwright) is an [Artillery engine](https://www.artillery.io/docs/reference/engines) lets you combine Playwright with Artillery to launch real browsers for full browser load testing.

## Requirements

**Tracetest Account**:

- Sign up to [`app.tracetest.io`](https://app.tracetest.io) or follow the [get started](/getting-started/overview) docs.
- Have access to the environment's [agent API key](https://app.tracetest.io/retrieve-token).

**Docker**: Have [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) installed on your machine.

## Run This Quckstart Example

The example below is provided as part of the Tracetest GitHub repo. You can download and run the example by following these steps:

Clone the Tracetest project and go to the Artillery + Playwright Quickstart:

```bash
git clone https://github.com/kubeshop/tracetest
cd tracetest/examples/quick-start-artillery-playwright
```

Follow these instructions to run the included demo app and TypeScript example:

1. Copy the `.env.template` file to `.env`.
2. Fill out the [TRACETEST_TOKEN and ENVIRONMENT_ID](https://app.tracetest.io/retrieve-token) details by editing your `.env` file.
3. Run `docker compose run tracetest-run`.
4. Follow the links in the output to view the test results.

Follow along with the sections below for an in detail breakdown of what the example you just ran did and how it works.

## Project Structure

The quick start Artillery + Playwright project is built with Docker Compose.

The [Pokeshop Demo App](/live-examples/pokeshop/overview) is a complete example of a distributed application using different back-end and front-end services. We will be launching it and running tests against it as part of this example.

The `docker-compose.yaml` file in the root directory of the quick start runs the Pokeshop Demo app, the OpenTelemetry Collector, Jaeger, and the [Tracetest Agent](/concepts/agent) setup.

The Artillery Plugin quick start has two primary files:

- Playwright Test Script file `resources/test.ts` that includes the `@tracetest/playwright` npm package and the Tracetest test definition. To view the Tracetest test definition as a YAML file you can also see the `import-pokemon.yaml` file.
- Artillery Playwright engine file `resources/artillery-playwright.yaml` that defines that the Artillery test will use the Playwright engine.

### The `resources/test.ts` Test Script

The `resources/test.ts` test script contains one test based on the Pokeshop Demo UI features:

- Import a Pokemon (using an async process)

### Tracetest Library Setup

If you go to the `package.json` file you will find the inclusion of a Tracetest package for Playwright `@tracetest/playwright`.

The test script imports the package, grabs the Tracetest API token from the environment variables and creates the Tracetest instance. It then defines the `importPokemon` test and uses the Tracetest test definition YAML.

The complete test script looks like this:

```typescript title=resources/test.ts
import { Page } from 'playwright';
import { expect, TestInfo } from '@playwright/test';
import Tracetest from '@tracetest/playwright';
import { config } from 'dotenv';

config();

const { TRACETEST_TOKEN = '', TRACETEST_ENVIRONMENT_ID = '' } = process.env;

const definition = `
type: Test
spec:
  id: artillery-playwight-import-pokemon
  name: "Artillery Playwright - Import Pokemon"
  trigger:
    type: playwright
  specs:
    - selector: span[tracetest.span.type="general" name = "validate request"] span[tracetest.span.type="http"]
      name: "All HTTP Spans: Status  code is 200"
      assertions:
        - attr:http.status_code = 200
    - selector: span[tracetest.span.type="http" name="GET" http.method="GET"]
      assertions:
        - attr:http.route = "/api/v2/pokemon/\${var:POKEMON_ID}"
    - selector: span[tracetest.span.type="database"]
      name: "All Database Spans: Processing time is less than 1s"
      assertions:
        - attr:tracetest.span.duration < 1s
  outputs:
    - name: DATABASE_POKEMON_ID
      selector: span[tracetest.span.type="database" name="create postgres.pokemon" db.system="postgres" db.name="postgres" db.user="postgres" db.operation="create" db.sql.table="pokemon"]
      value: attr:db.result | json_path '$.id'
`;

export async function importPokemon(page: Page) {
  const tracetest = await Tracetest({ apiToken: TRACETEST_TOKEN, environmentId: TRACETEST_ENVIRONMENT_ID });
  const title = 'Artillery Playwright - Import Pokemon';
  const pokemonId = Math.floor(Math.random() * 101).toString();
  await page.goto('/');

  await tracetest?.setOptions({
    [title]: {
      definition,
      runInfo: {
        variables: [
          {
            key: 'POKEMON_ID',
            value: pokemonId,
          },
        ],
      },
    },
  });

  await tracetest?.capture(page, { title: 'Artillery Playwright - Import Pokemon', config: {} } as TestInfo);

  expect(await page.getByText('Pokeshop')).toBeTruthy();

  await page.click('text=Import');

  await page.getByLabel('ID').fill(pokemonId);
  await page.getByRole('button', { name: 'OK', exact: true }).click();

  await tracetest?.summary();
}
```

## Creating the Artillery Test Script

The `resources/artilery-playwright.yaml` file contains the Artillery Test Script that will be used to trigger requests against the Pokeshop Demo and run trace-based tests. The steps executed by this script are the following:

1. Defines a scenario using the `playwright` engine running the `importPokemon` from `resources/test.ts` which is a Playwright test using the Tracetest integration using a random number as the Pokemon ID.
2. Defines the target as the Pokeshop Demo App running on `http://localhost:8081`.

```yaml title="resources/artilery-playwright.yaml"
config:
  target: http://localhost:8081
  engines:
    playwright: {}
  processor: test.ts
scenarios:
  - engine: playwright
    testFunction: importPokemon
```

## Setting the Environment Variables

Copy the `.env.template` file to `.env` and add the Tracetest API token and agent tokens to the `TRACETEST_TOKEN` and `TRACETEST_ENVIRONMENT_ID` variables.

## Running the Full Example

To start the Pokeshop Demo App, run the following command from the root directory:

```bash
docker compose run tracetest-run
```

## Finding the Results

The output from the Tracetest Engine script should be visible in the console log after running the test command. This log will show links to Tracetest for each of the test runs invoked by the Artillery Testing Script. Click a link to launch Tracetest and view the test result.

```bash title=Output
[+] Running 2/2
 ✔ worker Pulled                                                                                                                                                                     0.9s
 ✔ api Pulled                                                                                                                                                                        0.9s
[+] Building 0.7s (10/10) FINISHED                                                                                                                                   docker:desktop-linux
 => [tracetest-apply internal] load build definition from Dockerfile.tracetest                                                                                                       0.0s
 => => transferring dockerfile: 336B                                                                                                                                                 0.0s
 => [tracetest-apply internal] load metadata for docker.io/library/alpine:latest                                                                                                     0.6s
 => [tracetest-apply auth] library/alpine:pull token for registry-1.docker.io                                                                                                        0.0s
 => [tracetest-apply internal] load .dockerignore                                                                                                                                    0.0s
 => => transferring context: 2B                                                                                                                                                      0.0s
 => [tracetest-apply 1/5] FROM docker.io/library/alpine:latest@sha256:0a4eaa0eecf5f8c050e5bba433f58c052be7587ee8af3e8b3910ef9ab5fbe9f5                                               0.0s
 => CACHED [tracetest-apply 2/5] WORKDIR /app                                                                                                                                        0.0s
 => CACHED [tracetest-apply 3/5] RUN apk --update add bash jq curl                                                                                                                   0.0s
 => CACHED [tracetest-apply 4/5] RUN curl -L https://raw.githubusercontent.com/kubeshop/tracetest/main/install-cli.sh | bash -s -- v1.4.0                                            0.0s
 => CACHED [tracetest-apply 5/5] WORKDIR /resources                                                                                                                                  0.0s
 => [tracetest-apply] exporting to image                                                                                                                                             0.0s
 => => exporting layers                                                                                                                                                              0.0s
 => => writing image sha256:746b8de43a912dc68c81345fd5b2a54740f3d8b0b73724dc79a31abea045d748                                                                                         0.0s
 => => naming to docker.io/library/pokeshop-tracetest-apply                                                                                                                          0.0s
[+] Creating 10/8
 ✔ Network pokeshop_default              Created                                                                                                                                     0.0s
 ✔ Container pokeshop-postgres-1         Created                                                                                                                                     0.2s
 ✔ Container pokeshop-jaeger-1           Created                                                                                                                                     0.2s
 ✔ Container pokeshop-queue-1            Created                                                                                                                                     0.2s
 ✔ Container pokeshop-cache-1            Created                                                                                                                                     0.2s
 ✔ Container pokeshop-otel-collector-1   Created                                                                                                                                     0.1s
 ✔ Container pokeshop-worker-1           Created                                                                                                                                     0.1s
 ✔ Container pokeshop-api-1              Created                                                                                                                                     0.0s
 ✔ Container pokeshop-tracetest-agent-1  Created                                                                                                                                     0.0s
 ✔ Container pokeshop-tracetest-apply-1  Created                                                                                                                                     0.1s
[+] Running 9/9
 ✔ Container pokeshop-queue-1            Healthy                                                                                                                                    12.2s
 ✔ Container pokeshop-jaeger-1           Healthy                                                                                                                                     1.9s
 ✔ Container pokeshop-cache-1            Healthy                                                                                                                                    12.2s
 ✔ Container pokeshop-postgres-1         Healthy                                                                                                                                    12.2s
 ✔ Container pokeshop-otel-collector-1   Started                                                                                                                                     0.4s
 ✔ Container pokeshop-worker-1           Started                                                                                                                                     0.1s
 ✔ Container pokeshop-api-1              Healthy                                                                                                                                     1.6s
 ✔ Container pokeshop-tracetest-agent-1  Started                                                                                                                                     0.1s
 ✔ Container pokeshop-tracetest-apply-1  Started                                                                                                                                     0.1s
[+] Running 2/2
 ✔ api Pulled                                                                                                                                                                        0.8s
 ✔ worker Pulled                                                                                                                                                                     0.8s

> quick-start-artillery-playwright@1.0.0 test
> artillery run resources/artillery-playwright.yaml

Test run id: twmey_ngwbckeqpwm9ja9p99hqhfjtbhefp_bw8c
Bundled Typescript file into JS. New processor path: /app/resources/dist/test.js
Phase started: unnamed (index: 0, duration: 1s) 16:55:42(+0000)

Phase completed: unnamed (index: 0, duration: 1s) 16:55:43(+0000)

--------------------------------------
Metrics for period to: 16:55:50(+0000) (width: 2.21s)
--------------------------------------

browser.http_requests: ......................................................... 10
browser.page.FCP.http://api:8081/:
  min: ......................................................................... 175.3
  max: ......................................................................... 175.3
  mean: ........................................................................ 175.3
  median: ...................................................................... 175.9
  p95: ......................................................................... 175.9
  p99: ......................................................................... 175.9
browser.page.FID.http://api:8081/:
  min: ......................................................................... 1.6
  max: ......................................................................... 1.6
  mean: ........................................................................ 1.6
  median: ...................................................................... 1.6
  p95: ......................................................................... 1.6
  p99: ......................................................................... 1.6
browser.page.LCP.http://api:8081/:
  min: ......................................................................... 193.4
  max: ......................................................................... 193.4
  mean: ........................................................................ 193.4
  median: ...................................................................... 194.4
  p95: ......................................................................... 194.4
  p99: ......................................................................... 194.4
browser.page.TTFB.http://api:8081/:
  min: ......................................................................... 14.9
  max: ......................................................................... 14.9
  mean: ........................................................................ 14.9
  median: ...................................................................... 15
  p95: ......................................................................... 15
  p99: ......................................................................... 15
browser.page.codes.200: ........................................................ 10
vusers.created: ................................................................ 1
vusers.created_by_name.0: ...................................................... 1


⠴ Run Group: #e2fae28c-ac06-4001-8dc1-2328eee32e5a (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments//run/e2fae28c-ac06-4001-8dc1-2328eee32e5a)
Failed: 0
Succeed: 1
Pending: 0



Runs:

✔ Artillery Playwright - Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments//test/artillery-playwight-import-pokemon/run/7) - trace id: 1b3ba6a22f595dbfad1c5d32120aad20

--------------------------------------
Metrics for period to: 16:56:00(+0000) (width: 0s)
--------------------------------------

vusers.completed: .............................................................. 1
vusers.failed: ................................................................. 0
vusers.session_length:
  min: ......................................................................... 15189.2
  max: ......................................................................... 15189.2
  mean: ........................................................................ 15189.2
  median: ...................................................................... 15218.6
  p95: ......................................................................... 15218.6
  p99: ......................................................................... 15218.6


All VUs finished. Total time: 16 seconds

--------------------------------
Summary report @ 16:56:00(+0000)
--------------------------------

browser.http_requests: ......................................................... 10
browser.page.FCP.http://api:8081/:
  min: ......................................................................... 175.3
  max: ......................................................................... 175.3
  mean: ........................................................................ 175.3
  median: ...................................................................... 175.9
  p95: ......................................................................... 175.9
  p99: ......................................................................... 175.9
browser.page.FID.http://api:8081/:
  min: ......................................................................... 1.6
  max: ......................................................................... 1.6
  mean: ........................................................................ 1.6
  median: ...................................................................... 1.6
  p95: ......................................................................... 1.6
  p99: ......................................................................... 1.6
browser.page.LCP.http://api:8081/:
  min: ......................................................................... 193.4
  max: ......................................................................... 193.4
  mean: ........................................................................ 193.4
  median: ...................................................................... 194.4
  p95: ......................................................................... 194.4
  p99: ......................................................................... 194.4
browser.page.TTFB.http://api:8081/:
  min: ......................................................................... 14.9
  max: ......................................................................... 14.9
  mean: ........................................................................ 14.9
  median: ...................................................................... 15
  p95: ......................................................................... 15
  p99: ......................................................................... 15
browser.page.codes.200: ........................................................ 10
vusers.completed: .............................................................. 1
vusers.created: ................................................................ 1
vusers.created_by_name.0: ...................................................... 1
vusers.failed: ................................................................. 0
vusers.session_length:
  min: ......................................................................... 15189.2
  max: ......................................................................... 15189.2
  mean: ........................................................................ 15189.2
  median: ...................................................................... 15218.6
  p95: ......................................................................... 15218.6
  p99: ......................................................................... 15218.6
```

## What's Next?

After running the test, you can click the run link for any of them, update the assertions, and run the scripts once more. This flow enables complete a trace-based TDD flow.

![assertions](https://res.cloudinary.com/djwdcmwdz/image/upload/v1709916703/docs/app.tracetest.io_organizations_ttorg_e66318ba6544b856_environments_ttenv_956484fe2078461e_tests_ma21ae.png)

## Learn More

Please visit our [examples in GitHub](https://github.com/kubeshop/tracetest/tree/main/examples) and join our [Slack Community](https://dub.sh/tracetest-community) for more info!
